-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow setting database name and user when using JDBC URL #594
Allow setting database name and user when using JDBC URL #594
Conversation
…ainers#593) * Mark DockerMachineClientProviderStrategy as not persistable * Update CHANGELOG.md
I did not see those two earlier. For filtering out TC parameters, we could do this -
WDYT? That makes me think if TC_PARAM_MATCHING_PATTERN should only consider Pattern to include parameters starting with |
@rnorth Codacy is complaining to have atleast one JUnit assert but ignoring the VisibleAssertions that I have in test cases. I hope that is ok ;). |
…ainers-java into feature/TC_ISSUE_566
This PR has a lot of Spock changes showing when I view the diff. Has something strange happened with the branches? 😄 |
assertEquals("Database name from URL String is used.", "databasename", resultDB); | ||
return true; | ||
}); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is just a trivial comment, but please could you reformat using 4 spaces for indentation? IntelliJ/Eclipse defaults ought to be reasonably close. I'll add automatic formatting soon...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, I'll do that. Thanks for reviewing!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have updated formatting in recent commit. Thanks!
Hm, could it be because I merged latest upstream changes from the master? I thought that would make it easy to merge (if appropriate) :) |
Yeah, it looks like that’s what’s happened. Very odd. I’ll try and review properly later today!
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is good work - thank you for doing this, and I apologise that you've had to refactor quite a few bits along the way. I have a few comments but nothing massive!
@Getter | ||
private Optional<String> queryString; | ||
|
||
private ConnectionUrl() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is redundant, given that there is a public constructor with a parameter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has been refactored along with adding a static factory method to instantiate the class.
return url.startsWith("jdbc:tc:"); | ||
} | ||
|
||
public void parseUrl() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like this is only used together with the constructor. Could we instead have a static factory method that instantiates, parses and returns?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense. I have added a static method newInstance(final String url)
that returns ready to use instance of ConnectionUrl.
} | ||
|
||
Matcher daemonMatcher = Patterns.DAEMON_MATCHING_PATTERN.matcher(this.getUrl()); | ||
inDaemonMode = daemonMatcher.matches() ? Boolean.parseBoolean(daemonMatcher.group(2)) : false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IntelliJ is telling me that this can be simplified to:
inDaemonMode = daemonMatcher.matches() && Boolean.parseBoolean(daemonMatcher.group(2));
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has been refactored now.
* | ||
* @return {@link Map} | ||
*/ | ||
public Map<String, String> getQueryParameters() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like this method mutates the queryString
field as a side effect of constructing the Map. If someone calls getQueryString
too early, they'll get nothing.
Please could we avoid this? Maybe we could make all fields be final (which a static factory method and maybe Lombok could help with)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have modified the query and container parameter extraction method to make them immutable. Now, queryParameters
and containerParameters
have getters that returns unmodifiable maps and queryString
is instantiated only once during initial parsing. Please check.
public interface Patterns { | ||
final Pattern URL_MATCHING_PATTERN = Pattern.compile("jdbc:tc:([a-z]+)(:([^:]+))?://([^\\?]+)(\\?.*)?"); | ||
|
||
final Pattern ORACLE_URL_MATCHING_PATTERN = Pattern.compile("jdbc:tc:([a-z]+)(:([^(thin:)]+))?:thin:@([^\\?]+)(\\?.*)?"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I apologise for putting you through the pain of working with these regular expressions
😂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, I liked it. I never used these many and types of regex pattern, got to learn new things ;), So Thank you!
@@ -80,51 +69,34 @@ public synchronized Connection connect(String url, final Properties info) throws | |||
if (!acceptsURL(url)) { | |||
return null; | |||
} | |||
|
|||
ConnectionUrl cUrl = new ConnectionUrl(url); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please could you call this parameter connectionUrl
? Maybe it's just me, but every time I see cUrl
I think of this. In general we're quite OK with medium-verbosity names for things in this project 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed now 😄
/** | ||
* Factory for MySQL containers. | ||
*/ | ||
public class MySQLContainerProvider extends JdbcDatabaseContainerProvider { | ||
|
||
/** | ||
* Groups URL of format "jdbc:tc:(databaseType):(optinal_image_tag)//(hostanme)(optional :(numeric_port))/(databasename)(?parameters)" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple of typos in here (optinal
and hostanme
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typos are corrected now.
Matcher urlMatcher = MYSQL_URL_MATCHING_PATTERN.matcher(url.getUrl()); | ||
|
||
if(!urlMatcher.matches()) { | ||
//TODO: Is this necessary? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably not necessary if the ConnectionUrl.parseQuery()
method has been called..?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree. I have removed this matcher related code.
@@ -0,0 +1,48 @@ | |||
# TestContainers-Spock |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No idea why these Spock files are showing up in the PR - looking at the branch locally I don't see this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure what should I do for this one. Maybe it can be ignored while merging, if appropriate! I don't have any changes for Spock 😄
CHANGELOG.md
Outdated
@@ -9,6 +9,7 @@ All notable changes to this project will be documented in this file. | |||
### Changed | |||
- Abstracted and changed database init script functionality to support use of SQL-like scripts with non-JDBC connections. ([\#551](https://github.com/testcontainers/testcontainers-java/pull/551)) | |||
- Added `JdbcDatabaseContainer(Future)` constructor. ([\#543](https://github.com/testcontainers/testcontainers-java/issues/543)) | |||
- Mark DockerMachineClientProviderStrategy as not persistable ([\#593](https://github.com/testcontainers/testcontainers-java/pull/593)) | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please could you update the changelog to describe this change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have updated Changelog with items that I feel related to this PR. Please take a look and let me know/feel free to modify as you seem right.
JFYI Unless this PR integrated there's problem in |
Thank you very much for the review @rnorth. I will check these comments and make appropriate changes.
…
On Mar 13, 2018 at 4:33 PM, <Richard North ***@***.***)> wrote:
@rnorth requested changes on this pull request.
This is good work - thank you for doing this, and I apologise that you've had to refactor quite a few bits along the way. I have a few comments but nothing massive!
In modules/jdbc/src/main/java/org/testcontainers/jdbc/ConnectionUrl.java (#594 (comment)):
> + @getter + private Optional<Integer> databasePort = Optional.empty(); + + @getter + private Optional<String> databaseName = Optional.empty(); + + @getter + private Optional<String> initScriptPath = Optional.empty(); + + @getter + private Optional<InitFunctionDef> initFunction = Optional.empty(); + + @getter + private Optional<String> queryString; + + private ConnectionUrl() {
I think this is redundant, given that there is a public constructor with a parameter.
In modules/jdbc/src/main/java/org/testcontainers/jdbc/ConnectionUrl.java (#594 (comment)):
> + + /** + * This is a part of the connection string that may specify host:port/databasename. + * It may vary for different clients and so clients can parse it as needed. + * + * @return + */ + public String getDbHostString() { + return Objects.requireNonNull(this.dbHostString, "Database Host String cannot be null. Have you called parseUrl() method?"); + } + + public static boolean accepts(final String url) { + return url.startsWith("jdbc:tc:"); + } + + public void parseUrl() {
It looks like this is only used together with the constructor. Could we instead have a static factory method that instantiates, parses and returns?
In modules/jdbc/src/main/java/org/testcontainers/jdbc/ConnectionUrl.java (#594 (comment)):
> + + queryString = Optional.ofNullable(urlMatcher.group(5)); + getQueryParameters(); + + Matcher matcher = Patterns.INITSCRIPT_MATCHING_PATTERN.matcher(this.getUrl()); + if (matcher.matches()) { + initScriptPath = Optional.ofNullable(matcher.group(2)); + } + + Matcher funcMatcher = Patterns.INITFUNCTION_MATCHING_PATTERN.matcher(this.getUrl()); + if (funcMatcher.matches()) { + initFunction = Optional.of(new InitFunctionDef(funcMatcher.group(2), funcMatcher.group(4))); + } + + Matcher daemonMatcher = Patterns.DAEMON_MATCHING_PATTERN.matcher(this.getUrl()); + inDaemonMode = daemonMatcher.matches() ? Boolean.parseBoolean(daemonMatcher.group(2)) : false;
IntelliJ is telling me that this can be simplified to:
inDaemonMode = daemonMatcher.matches() && Boolean.parseBoolean(daemonMatcher.group(2));
In modules/jdbc/src/main/java/org/testcontainers/jdbc/ConnectionUrl.java (#594 (comment)):
> + Matcher matcher = Patterns.TC_PARAM_MATCHING_PATTERN.matcher(this.getUrl()); + while (matcher.find()) { + String key = matcher.group(1); + String value = matcher.group(2); + results.put(key, value); + } + + return results; + } + + /** + * Get all Query paramters specified in the Connection URL after ?. This also includes TestContainers parameters. + * + * @return ***@***.*** Map} + */ + public Map<String, String> getQueryParameters() {
It looks like this method mutates the queryString field as a side effect of constructing the Map. If someone calls getQueryString too early, they'll get nothing.
Please could we avoid this? Maybe we could make all fields be final (which a static factory method and maybe Lombok could help with)?
In modules/jdbc/src/main/java/org/testcontainers/jdbc/ConnectionUrl.java (#594 (comment)):
> + + @OverRide + public boolean equals(Object obj) { + if (Objects.isNull(obj) || !(obj instanceof ConnectionUrl)) return false; + return this.getUrl().equals(((ConnectionUrl) obj).getUrl()); + } + + /** + * This interface defines the Regex Patterns used by ***@***.*** ConnectionUrl}. + * + * @author manikmagar + */ + public interface Patterns { + final Pattern URL_MATCHING_PATTERN = Pattern.compile("jdbc:tc:([a-z]+)(:([^:]+))?://([^\\?]+)(\\?.*)?"); + + final Pattern ORACLE_URL_MATCHING_PATTERN = Pattern.compile("jdbc:tc:([a-z]+)(:([^(thin:)]+))?:thin:@([^\\?]+)(\\?.*)?");
I apologise for putting you through the pain of working with these regular expressions
😂
In modules/jdbc/src/main/java/org/testcontainers/jdbc/ContainerDatabaseDriver.java (#594 (comment)):
> @@ -80,51 +69,34 @@ public synchronized Connection connect(String url, final Properties info) throws if (!acceptsURL(url)) { return null; } + + ConnectionUrl cUrl = new ConnectionUrl(url);
Please could you call this parameter connectionUrl? Maybe it's just me, but every time I see cUrl I think of this (https://curl.haxx.se/). In general we're quite OK with medium-verbosity names for things in this project 😄
In modules/mysql/src/main/java/org/testcontainers/containers/MySQLContainerProvider.java (#594 (comment)):
> /** * Factory for MySQL containers. */ public class MySQLContainerProvider extends JdbcDatabaseContainerProvider { + + /** + * Groups URL of format "jdbc:tc:(databaseType):(optinal_image_tag)//(hostanme)(optional :(numeric_port))/(databasename)(?parameters)"
A couple of typos in here (optinal and hostanme)
In modules/mysql/src/main/java/org/testcontainers/containers/MySQLContainerProvider.java (#594 (comment)):
> @@ -13,4 +30,26 @@ public boolean supports(String databaseType) { public JdbcDatabaseContainer newInstance(String tag) { return new MySQLContainer(MySQLContainer.IMAGE + ":" + tag); } + + @OverRide + public JdbcDatabaseContainer newInstance(ConnectionUrl url) { + Objects.requireNonNull(url, "Connection URL cannot be null"); + + Matcher urlMatcher = MYSQL_URL_MATCHING_PATTERN.matcher(url.getUrl()); + + if(!urlMatcher.matches()) { + //TODO: Is this necessary?
Probably not necessary if the ConnectionUrl.parseQuery() method has been called..?
In modules/spock/README.md (#594 (comment)):
> @@ -0,0 +1,48 @@ +# TestContainers-Spock
No idea why these Spock files are showing up in the PR - looking at the branch locally I don't see this.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub (#594 (review)), or mute the thread (https://github.com/notifications/unsubscribe-auth/AA1i5hZU68S_58BNYdOIZ_ENf63Wmqs_ks5teC0JgaJpZM4SSr4s).
|
@VladRassokhin Thanks for that info. I did add a test and verified behavior. It is working as expected with changes in this PR. Should there be a Bug entry created and referenced here? Anyways, I'll update the PR description too. |
@manikmagar I think no bug entry needed if this PR would be merged prior to any new release |
@rnorth I have made changes to take care of review comments, please take a look when you get chance. Thanks for reviewing. @VladRassokhin I did make changes to a test case that would fail if query parameters are not used when creating connection from cached container - Please See this. |
could use a formatter run over this. lots if apparent diffs are just formatting changes, and the rest aren't consistent. makes it hard to see what has really changed. |
Thanks @jeacott for looking through these changes. I tried to use the same formatting as rest of the code, which is pretty much standard intellij code formatting. Could you please point me to some code where you see it is not appropriate? I'll be happy to revisit and correct appropriately. |
I think there is something off with this PR, showing a lot of changes that aren't changes but have been introduced simply by rebasing to master or something (i.e. the changes in the Oh, I just see @rnorth has already mentioned this :D |
@Override | ||
public boolean acceptsURL(String url) throws SQLException { | ||
return url.startsWith("jdbc:tc:"); | ||
return url.startsWith("jdbc:tc:"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
formatting in this file is inconsistent with the other 4 space indents etc.
if (candidateContainerType.supports(databaseType)) { | ||
container = candidateContainerType.newInstance(tag); | ||
if (candidateContainerType.supports(connectionUrl.getDatabaseType())) { | ||
container = candidateContainerType.newInstance(connectionUrl); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this mod to connectionUrl instead of 'tag' intentional?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it is intentional and part of the implementation. Please see PR Description for details.
databaseName = Optional.of(dbInstanceMatcher.group(4)); | ||
} | ||
|
||
queryParameters = Collections.unmodifiableMap( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this doesnt allow for duplicate params which might be very handy if for example I want to set jdbc://...?TC_INITSCRIPT=schema.sql&TC_INITSCRIPT=data.sql
regular expressions seem like a hard work way to parse a jdbc url.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see what you mean but I don't think it is possible even with the existing implementation. I would rather think of that as a separate feature request, maybe @rnorth can weigh in.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please let's see if we can tackle this without regular expressions separately. I'd be very happy if we could do that and make this code easier to read.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just making sure I understand it correctly. Do You mean, 'Allowing multiple TC_INITSCRIPT like parameter support' should be implemented in separate PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On other thought, maybe a comma delimited list of files is accepted for single TC_INITSCRIPT parameter and executed in the sequence they are provided. Eg. TC_INITSCRIPT=Script1.sql,Script2.sql,Script3.sql
.
@@ -93,7 +97,10 @@ public static DockerClientProviderStrategy getFirstValidStrategy(List<DockerClie | |||
LOGGER.warn("Can't instantiate a strategy from {}", it, e); | |||
return Stream.empty(); | |||
} | |||
}), | |||
}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is the formatting in this Stream.concat... block just 'idea' being weird? doesnt seem to follow the 4 space indents everywhere else.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This class is not modified and related for this PR, probably upstream has changed and so you see the difference. I would leave it as-is for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please let's see if we can tackle this without regular expressions separately. I'd be very happy if we could do that and make this code easier to read.
Response to wrong comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I originally thought this looked worse than it is.
@manikmagar could you please try rebasing onto master and force-push this branch to see if this clears the diff noise? I've pulled the branch locally and the commits look fine, so I find it a bit disconcerting that we're all seeing phantom changes in the PR. I'm not sure I trust Github to merge it well if it can't show the diff correctly... |
Ok @rnorth , let me try rebasing onto master. In the worst case, I can create a new PR with related changes only. |
I think something is really weird happened with this branch, I will try to create a new branch and re-submit the clean PR on the latest upstream master. |
@manikmagar Thanks a lot for the new PR, I'll close this one, so we don't get confused. |
This PR implements #566.
Note: Other Containers Providers such as PostgreSQL will remain unaffected and continue to use earlier implementation of newInstance(tag) due to default delegation, until another implementation is provided.